This relates to apparatus for parallel processing of signals.
Recently, advances in the computational circuits art have brought to the forefront a class of highly parallel computation circuits that solve a large class of complex problems in analog fashion. These circuits comprise a plurality of amplifiers having a sigmoid transfer function and a resistive feedback network that connects the output of each amplifier to the input of the other amplifiers. Each amplifier input also includes a capacitor connected to ground and a resistor connected to ground, which may or may not include a capacitance and a conductance in addition to the parasitic capacitance and conductance. Input currents are fed into each amplifier input, and output is obtained from the collection of output voltages of the amplifiers.
A generalized diagram of this circuit is shown in FIG. 1, depicting amplifiers 10, 11, 12, 13 and 14 with positive and negative outputs V.sub.1, V.sub.2, V.sub.3, V.sub.4, and V.sub.n, respectively. Those outputs are connected to an interconnection block 20 which has output lines 41-45 connected to the input ports of amplifiers 10-14, respectively. Within interconnection block 20, each output voltage V.sub.i is connected to each and every output line of block 20 through a conductance (e.g., resistor). For convenience, the conductance may be identified by the specific output line (i.e., source) that is connected by the conductance to a specific voltage line. For example, T.sub.21.sup.+ identifies the conductance that connects the non-inverting output V.sub.2 to the input of the first amplifier (line 41).
Also connected to each amplifier input port is a parallel arrangement of a resistor and a capacitor with their second lead connected to grounded, and means for injecting a current (from some outside source) into each input port.
Applying Kirchoff's current law to the input port of each amplifier i of FIG. 1 yields the equation: ##EQU1## where C.sub.i is the capacitance between the input of amplifier i and ground,
1/R.sub.i is the equivalent resistance and it equals ##EQU2## where .rho..sub.i is the resistance between the input of amplifier i and ground, PA0 U.sub.i is the voltage at the input of amplifier i, PA0 T.sub.ij.sup.+ is the a conductance between the non-inverting output of amplifier j and the input of amplifier i, PA0 T.sub.ij.sup.- is the a conductance between the inverting output of amplifier j and the input of amplifier i, PA0 V.sub.j is the positive output voltage of amplifier j, related U.sub.j by the equation V.sub.j =g.sub.i (U.sub.j), and PA0 I.sub.i is the current driven into the input port of amplifier i by an external source.
When T.sub.ij.sup.+ and T.sub.ij.sup.- are disjoint, T.sub.ij.sup.+ -T.sub.ij.sup.- may for convenience be expressed as T.sub.ij, and it is well known that a circuit satisfying Equation (1) with symmetric T.sub.ij terms is stable. It is also well known that such a circuit responds to applied stimuli, and reaches a steady state condition after a short transition time. At steady state, dU.sub.i /dt=0 and dV.sub.i /dt=0.
With this known stability in mind, the behavior of other functions may be studied which relate to the circuit of FIG. 1 and involve the input signals of the circuit, the output signals of the circuit, and/or the circuit's internal parameters.
Indeed, in a copending application entitled Optimization Network for the Decomposition of Signals, by J. J. Hopfield, a function was studied that has the form ##EQU3## It is observed in this copending application that the integral of the function g.sub.i.sup.-1 (V) approaches 0 as the gain of amplifier i approaches infinity. It is also shown in the Hopfield application that the time derivative of the function E is negative, and that it reaches 0 when the time derivative of voltages V.sub.i reaches 0. Since equation (1) assures the condition of dV.sub.i /dt approaching 0 for all i, the function E of equation (2) is assured of reaching a stable state. The discovery of this function E led to the use of the FIG. 1 circuit in problem solving applications (such as the classic traveling salesman problem), in associative memory applications, and in decomposition problems (as disclosed in another copending application, by J. J. Hopfield and D. W. Tank, titled Optimization Network for the Decomposition of Signals).
The FIG. 1 circuit can solve problems which can be structured so that the the function to be minimized has at most second order terms in some parameters of the problem to permit correspondence to Equation (2). Other problems, however, may required the minimization of equations that contain terms of order higher than two. Those problems are solvable through the use of inter-neuron amplifiers, as described in still another copending application filed on even data herewith, entitled "A Highly Parallel Computation Network With Means for Reducing the Algebraic Degree of the Objective Function".
In each one of above, the T.sub.ij.sup.+ and the T.sub.ij.sup.- are conductances that assume different values, which are a function of the problem to be solved. But, in some applications mathematical niceties take a back seat to physical realities. In particular, in most physical systems, there is a relatively limited dynamic range for the values that T.sub.ij can conveniently assume. It is important to be able to construct these highly parallel interactive circuits with a limited repertoire of T.sub.ij conductances. An additional substantial benefit would result from the use of only two-valued conductances in that the amplifiers employed in the neural network can dispense with the positive output altogether. This would permit the use of amplifiers that are smaller, and allow the construction of the neural networks with fewer leads. The latter benefit would be substantial because it is the number of leads to and from the conductance matrix that, most often, is the limiting factor in the construction of large neural networks.